Unifying Event Detection and Captioning as Sequence Generation via Pre-training

Dense video captioning aims to generate corresponding text descriptions for a series of events in the untrimmed video, which can be divided into two sub-tasks, event detection and captioning. Unlike previous works that tackle sub-tasks separately, recent have focused on enhancing inter-task association between sub-tasks. However, designing interactions is not trivial due large differences their task specific solutions. Besides, methods normally ignore temporal dependencies events, leading redundancy or inconsistency problems. To above defects, this paper, we define as sequence generation propose unified pre-training fine-tuning framework naturally enhance Since model predicts each with context, inter-dependency fully exploited thus our detect more diverse consistent video. Experiments ActivityNet dataset show outperforms state-of-the-art methods, further boosted when pre-trained extra large-scale video-text data. Code available at https://github.com/QiQAng/UEDVC .